A prior art general monitoring apparatus is generally configured so that a monitoring objective part in a shop and the like is shot by a camera or plural cameras and that the image(s) is displayed on a monitor screen. In this case, for example, when plural cameras are installed, the same number of monitor screens as the number of said cameras are generally prepared. When the same number of monitor screens as the number of the cameras can not be prepared, the images of said plural cameras are integrated into a single image using a separator and the like, the camera images are sequentially switched, or another method is used. However, there has been such a problem or the like that a manager needs to consider the continuity between the independently displayed images during the monitoring of the image of each camera in such a prior art apparatus.
To resolve the problem, for example, Japanese Laid-Open Patent Publication No. Hei 10-164566, etc. discloses a monitoring apparatus in which a plurality of cameras are installed so that the images thereof overlap at the edge parts and in which the images of the plurality of cameras are integrated and displayed into a single image by superposing said overlapping parts, thereby resolving said problem of continuity.
Another example of the application of a monitoring apparatus is the case of the installation on a vehicle. One of the prior art example is as follows. That is a monitoring apparatus in which a camera is installed for monitoring the surroundings of a vehicle and the image acquired by said camera is displayed on a TV monitor installed near the driver's seat. An apparatus is known in which a place, such as the rear side of the vehicle, unseen to the driver with the own eyes or a mirror is shot by said camera and displayed on said monitor. In particular, many are used on a vehicle, such as a large truck and a van (one-box wagon), having a narrow field of view seen with the eyes and a mirror.
FIG. 69 shows a prior art example in which monitoring cameras are installed on a vehicle. The example of FIG. 69 is a method in which the images of four monitoring cameras (C1-C4) mounted on the body of the vehicle are synthesized into a single image through a separating adapter and the images are separately displayed on a TV monitor (D1-D4). In particular, with regard to the image of the rear side camera, an invention is made so that the right and left of the image is reversed in order that the positional relation of the objects viewed in the image becomes the same as viewed in a mirror. Further, with regard to a place unseen due to the limitation in the angle of field of the camera, an invention is made so that each camera is turned by manual operation by the driver thereby permitting to obtain the image of the desired place. Such a monitoring apparatus mentioned above is disclosed, for example, in Japanese Laid-Open Patent Publication No. Hei 5-310078, and so on.
However, in a prior art monitoring apparatus such as the above-mentioned example, the images of respective cameras are displayed independently of each other. Accordingly, it is difficult to understand instantaneously the overall situation of the space shot by the cameras by watching each image.
In addition, in case of synthesizing the respective images into a single image, the apparatus cannot treat, for example, a request by the manager to watch an integrated image viewed from a desired point of view depending on the situation, because the apparatus does not calculate the spatial position of an object shot by the camera.
The same problem occurs in a monitoring apparatus installed on a vehicle. That is, in a prior art monitoring apparatus such as the above-mentioned example, the images input from respective cameras are displayed independently of each other. Accordingly, for example, in case of using said displayed images as assistance during the parking of a vehicle in a parking space, said images merely serve to provide a view otherwise unseen. That is, the only effect is to reduce the area unseen to the driver in comparison with the conventional case by visual inspection and a mirror.
Meanwhile, a wide-angle lens is generally used in order to overcome the narrowness of the field. The image from the wide-angle lens does not provide the detail in a specific part. However, the overall situation of the surroundings of the vehicle is easily understood because of the wideness of the field of the wide-angle lens.
However, even if an ordinary lens is replaced by the wide-angle lens, the image of the camera still depends on the position on the body of a vehicle where the camera is installed, and accordingly, it is not possible to obtain a virtual image viewed from a position where the camera is not installed. That is, the only effect of using the camera equipped with the wide-angle lens is to reduce the number of the cameras.
Another prior art apparatus is further disclosed in Japanese Laid-Open Patent Publication No. Hei 3-99952. FIG. 70 is a block diagram showing an embodiment of the prior art apparatus for monitoring the surroundings of a vehicle. The images input from cameras 1-N 2201 into an image converting section 2202 are converted into another coordinates by a conversion, and synthesized into a single image by an image displaying section 2203. It is then displayed on a TV monitor 2204 installed at the driver's seat. In the image displaying section, an invention is possible in which the displayed position of the vehicle in question is shifted from the center of the screen or the like, depending on gear position, vehicle speed, and winker operation in order to widen the area of interest of the surroundings of the vehicle, and the like.
Further another prior art example developed further is disclosed in Japanese Laid-Open Patent Publication No. Hei 7-186833. In this example, in the presentation of the situation of the surroundings to the driver, road surface part is distinguished from the other part. Then, the road surface part is converted into an image viewed downward from the point of view above the vehicle center by a coordinate transformation. The part other than the road surface of the raw video from the camera is altered properly with regard to place and size, and displayed in superposition on said converted image. As such, it accurately informs the situation of an obstacle in the surroundings of the vehicle, especially another vehicle and the like approaching from behind the vehicle.
However, in the above-mentioned another prior art apparatus for monitoring the surroundings of a vehicle, there has been a problem that it can be difficult to find the correspondence of the obtained synthesized image to the actual object.
In the above-mentioned further another example, by separating the road surface part and the part other than the road surface, the viewed image of the objects outside of the road surface is cut out and pasted into a synthesized image. However, the cutting-out of the objects outside of the road surface is one of the difficult problems in the field of image recognition, and hence difficult to realize practically.
On the other hand, there is a problem that the shift of the camera during the running causes a shift in the synthesized image. A method for resolving this is disclosed, for example, in Japanese Laid-Open Patent Publication No. Hei 5-310078. In this method, a mechanism for changing the orientation of a camera is provided, whereby the driver manually directs the camera into a desired direction while viewing a synthesized image. The problem in this case is, however, that the mechanisms for changing the orientation are necessary to installed in the same number as the number of the cameras, which causes a higher cost.
To resolve such problems, in case of an apparatus installed on a vehicle for example, an object of the present invention is to compose a single image showing what objects are present near the vehicle in the all surroundings of the vehicle, as realistically and understandably as possible, thereby displaying it to the driver.
The present invention further provides a method for easily obtaining camera parameters, such as the attachment position and the attachment angle of the camera, and also an apparatus and a method for detecting and correcting the shift of said camera parameters due to vibration, temperature and the like, if it occurs during the running.